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Abstract —We study the problem of reconstructing a signal from 
its projection on a subspace. The proposed signal reconstruction 
algorithms utilize a guiding subspace that represents desired properties 
of reconstructed signals. We show that optimal reconstructed signals 
belong to a convex bonnded set, called the “reconstruction” set. 
We also develop iterative algorithms, based on conjugate gradient 
methods, to approximate optimal reconstructions with low memory 
and compntational costs. The effectiveness of the proposed approach 
is demonstrated for image magnification, where the reconstructed 
image quality is shown to exceed that of consistent and generalized 
reconstruction schemes. 

I. Introduction 

The problem of reconstructing a signal from its partial observa¬ 
tions is of fundamental importance in signal processing. A classical 
example is reconstruction of continuous bandlimited signals from 
their discrete time samples. There are numerous applications of 
signal reconstruction, e.g., image super-resolution (T), increasing 
audio frequency range (2)> snd semi-supervised learning a. 

We consider the problem of determining a reconstruction f of 
an original signal f from its measurement obtained by taking an 
orthogonal projection Sf onto a subspace S called the sampling 
subspace. Since sampling involves loss of information, we need 
some a priori assumptions on the original signal f to be recovered. 
One such assumption may be that the signal f belongs to a 
subspace T (of a vector space H) that can be thought of as a 
target reconstruction subspace. We prefer to call T a guiding 
reconstruction subspace, because, even though we expect f to have 
most of its energy contained in T, in our technique the reconstructed 
signal f is not necessarily restricted to T. The guiding subspace 
of the signal can be determined using some model of desirable 
reconstructed signal behavior. For example, it can be learned from 
a training dataset fl. For signals in euclidean spaces with natural 
spectral properties, a space of signals bandlimited in the transform 
domain (such as Fourier, cosine and wavelet) can be chosen as the 
guiding subspace. This idea can be extended to signals defined on 
manifolds or graphs, where T can be chosen as the space of smooth 
signals given by the linear combinations of first few eigenvectors of 
the Laplacian operator associated with the manifold or the graph j4l. 

A reconstruction f is said to be sample consistent if Sf = Sf, i.e., 
the measurements remain unchanged. A set of all signals, having the 
same samples Sf is a plane Sf-f 5^ that we call a sample consistent 
plane, where 5^ is the orthogonal complement to the sampling 
subspace S. The sets Sf -|-5^ and T in general may not intersect. 
In such a case, there is no sample consistent reconstruction which is 
also in T. For a solution, which is in both T and Sf-)-5^ to exist for 
any f, we need T+<S^ = T-L. Additionally, for such a solution to be 
unique we need Tn<S^ = {0}. If both the existence and uniqueness 
conditions are satisfied, then a unique sample consistent solution in 
T is given by Prxsf, where PrTS is an oblique projector on T 
along 5^ 0, m Non-uniqueness caused by T n 5^ 7^ {0} can 
be mathematically resolved by replacing H with a quotient space 


'HI{T n5^}. In practice, one can choose a unique solution by 
imposing additional constraints; see, e.g. 0. 

The assumption T -I- = % can be disadvantageous and very 

restrictive in applications. Even though it guarantees the existence 
of the intersection of Sf -1-5^ and T, finding this intersection 
numerically may be difficult as it is very sensitive to their mutual 
position (especially in high dimensions). It can lead to oblique 
projectors with large norms and make the reconstruction unstable. 

To counter this, oversampling is advocated (^, leading to a 
smaller consistent plane Sf -|- 5^ that may no longer intersect 
with T. In 0, the reconstructed signal is defined as a point in 
T having the smallest distance to Sf -f 5^ by enforcing the 
constraint f £ T and allowing f to be sample inconsistent. This is 
known as “generalized reconstruction”. Generalized reconstruction 
is more stable. However, the stability comes at the cost of potentially 
producing a sample inconsistent signal. In contrast, m describes 
a reconstruction method, which places the reconstructed signal in 
the sample consistent plane, relaxing the constraint that f £ T by 
minimizing the energy of the reconstruction in (the orthogonal 
complement of T). Thus, the reconstructed signal is a point in 
Sf -f having the smallest distance to T. This approach is mo¬ 
tivated by a realization that in practice it is hard to find a subspace 
T such that the signals of interest are completely contained in it. 
Thus, the subspace T is used as a guide, not as a true target, placing 
the trust on sampling. However, this may not be desirable, if the 
samples are noisy. 

In this paper, we provide a unified view of signal reconstruction 
under the oversampling scenario. In Section we define a set of 
reconstructions given by the convex combinations of generalized 
reconstruction and consistent reconstruction and show that it lies 
on the shortest pathway between the consistent hyperplane and the 
guiding subspace. A novel formulation of the sample consistent 
reconstruction in case of oversampling is provided in Section |I^ 
We present a conjugate gradient (CG) based iterative method to 
find the sample consistent reconstruction. Our formulation allows 
an implicit frame-less description of the guiding subspace via an 
action of the corresponding orthogonal projector T (that can be 
approximate) and also causes CG iterations to converge faster. 
This solution can then be projected on T to obtain generalized 
reconstruction. Section m connects our reconstruction set to a 
solution of a regularization-based reconstruction problem, cf. Col, 
that minimizes the weighted sum of a sample consistency and a 
smoothness term. We apply our approach to image magnification 
in Section [V] where the reconstructed image quality is shown to 
exceed that of consistent and generalized reconstructions. 

H. Reconstruction Set 

We consider the problem of reconstruction in the oversampling 
scenario. As stated before, the guiding set (or subspace) may not 
contain any sample consistent solutions. When the samples are 



Fig. 1: An example with dim'H = 3,dim5 = 2,dimT = 1 

noisy, the original signal does not lie in the sample consistent plane. 
On the other hand, the original signal may not be entirely contained 
in the guiding subspace either. In such a case, it is not clear which 
reconstruction, consistent (2) or generalized 0, ©, is better. This 
situation is illustrated in Fig. ^ by a simple geometric example, 
where dim^ = 3, dim 5 = 2 and dimT = 1. Here, the set of all 
signals having the same sample Sf is a line Sf + 5^. The lines 
Sf + and T in general do not intersect and no reconstruction 
f can be constrained to both lines as required in |5(, (6). 

We see in Fig. that the consistent reconstruction of can be 
viewed as an element from the consistent plane Sf + 5^ which 
minimizes the distance to the guiding subspace T. On the other 
hand, the generalized reconstruction of 0 is an element/rom the 
guiding subspace T, minimizing the distance to the consistent plane 
Sf + 5^. Clearly, the following equalities hold: 

min min||f —1|| = min ||f —1|| = min min ||f —1||. 

fesf+s^ te'T fgsf+5^ te'T fesr+s^ 

ter 

( 1 ) 

The above equations suggest defining a reconstruction set as a 
shortest pathway set between the consistent plane Sf + and 
the guiding subspace T. In Fig. [T] the reconstruction set is a line 
segment with the end points given by the consistent reconstmction 
and the generalized reconstruction. Any element of the reconstruc¬ 
tion set is a valid candidate for reconstruction when the sampling or 
guiding procedures are not known to be reliable. Section [Tvl shows 
an example of selecting an optimal solution from the reconstmction 
set when sampling is noisy and the amount of noise is known. 

III. Algorithm for finding the reconstruction set 

We now propose a novel algorithm for finding the sample con¬ 
sistent reconstruction (2, which relaxes the constraint that f G T 
and instead minimizes the energy in while maintaining sample 
consistency, as follows, 

inf ||f — Tf II subject to Sf = Sf. (2) 

f 

The above is equivalent to the problem 

inf /(x-FSf),T^ (x-FSf)V (3) 
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where x = f — Sf. If the solutions f and x to problems 0 
and 0, respectively, are not unique, we choose solutions in the 
corresponding factor spaces, e.g., the normal solution (i.e., with the 
smallest norm), to guarantee uniqueness. 

It can be shown that problem 0 is equivalent to solving 
Ax = b with A = (S^T-L) and b = -S^T^Sf. 
The notation (S^T^) makes it explicit that the domain of 


S^T^ is restricted to 5^. Conjugate gradient method (CG) is an 
optimal iterative method for solving Ax = b, when A is a linear 
self-adjoint non-negative operator with bounded (pseudo) inverse. 
Although is not self-adjoint in general, the restriction of 

to 5^ is self-adjoint. Therefore, we can use CG with 
A = (S^T^) 1^^ and b = —S^T^Sf. When the solution is 
not unique, CG converges to the unique normal solution x with 
minimum norm. Note that for CG to converge to the right solution, 
it must be initialized with some xq € 5^. 

In the special case when T + 5^ = H and T n 5^ = {0}, the 
solution f of 0 is the same as the result of the oblique projection 
Prusf in 0 , 0 . But our formulation and the resulting algorithms 
are different since they are based only on actions of orthogonal 
projectors T and S. The solution f of 0 is closely related to 
generalized reconstruction given by a signal in T that minimizes 
the distance to SfThis reconstruction is given by P 7 -xs{r)f> 
where Prxs{T) the oblique projector onto the subspace T along 
the orthogonal complement to the subspace S(T) C 5 0. Note 
that generalized reconstruction does not require T -\- — H 

(and thus, also allows for oversampling) but it may be sample 
inconsistent. As illustrated in Fig. [T] it is easy to show, using 
0, that P' 7 -xs(T)f — Tf. One can also show that, when the 
samples are noise free, reconstruction error is always better with 
the consistent least squares reconstruction given by 0 than with 
the generalized reconstruction, i.e. ||f — f|| < ||f — Tf||. We omit 
the details due to lack of space. 

Once the consistent reconstruction f G Sf -f 5^ and generalized 
reconstruction t = Tf G T are known, any element in the 
reconstruction set, defined in Section [0 can be obtained by taking 
their convex combination af -f (1 — a)Tf, where a G [0, 1]. If the 
samples are noise-free, we choose our reconstruction to be sample 
consistent, f G Sf -F5^. If there is noise in sample measurements, 
we may decide to trust the guiding subspace T more than the sample 
Sf and choose as our output reconstruction a convex combination 
af -I- (1 — a)Tf within the reconstruction set, where 0 < a < 1. 

IV. Relation to Regularized Reconstruction 

Regularization-based methods for reconstruction Go) can be 
formulated in our notation as the following unconstrained quadratic 
minimization problem 

inf ||Sf^-Sff-f p||Hf,f, p>0, (4) 

fp 

where the operator H can be thought of as a high pass filter, e.g., it 
may approximate our T^, in which case problem 0 approximates 

inf ||sfp - Sfll^ -f p II ffp - Tfp) 1^ . (5) 

ip " " 

Problem 0 can be viewed as a relaxation of 0 . We prove below 
that the set of all solutions of 0 for varying p > 0 is nothing but 
our reconstruction set (with end points removed). 

Theorem 1. Let the elements of the reconstruction set be given 
by the formula f^ = af -F (1 — a)Tf, where 0 < a < 1, and 
f solves 0 . Then the vector f^ is a solution of problem 0 with 
p = (1 - a)/a. 

Proof. On the one hand, minimization problem 0 is equivalent 
to the following linear equation (S -F pT^) fp = Sf. On the other 
hand, the consistent reconstruction f solves S^T^f = 0 and Sf = 



Sf. Taking p = (1 — a)/a and substituting for fp, we obtain by 
elementary calculations 

S + (af + (1 - a)Tf) = Sf 

using properties of S and T as projectors. ■ 

If there exists a unique intersection of the sample-consistent plane 
Sf -1-5^ and the guiding subspace T, as assumed in flOl . then this 
intersection f = Tf. Our reconstruction set is thus trivially reduced 
to this single element f = Tf. By Theorem the minimizer fj, in 
0 is simply fp = f = Tf, no matter what the value of p > 0 is. 

If our reconstruction set is non-trivial, we can move the recon¬ 
struction away from the sample-consistent plane Sf -I- 5^ toward 
the guiding subspace T. This may be beneficial when sampling 
is noisy. A specific value of the regularization parameter needs to 
be chosen a priori according to a noise level, if problem is 
solved directly. Theorem [T] allows us to circumvent this problem 
and choose p after determining the reconstruction set. Once we 
have f £ Sf -b 5^ and t = Tf £ T, we can compute the solution 
to 0 for a desired value of p by taking the convex combination 
af -b (1 — a)Tf with p = (1 — a)/a. This also allows us to try 
reconstructions with multiple values of p without solving 0 each 
time. Specifically, let the measurements be Sf -be, where e denotes 
the noise. If the noise energy ||e|| is known then we can select 


V. Simulations 

In this section, we present an application of the proposed recon¬ 
struction approach to the image magnification problem. 

A. Problem setting 

Let f be a high resolution image of size w x w. We assume that 
a sampled low resolution version of f is obtained by a sampling 
operator BJ which downsizes the image by a factor of r using rxr 
averaging and then downsampling. Its adjoint B 5 upsamples a low 
resolution image by simply copying each pixel value in a r x r block 
to get back ^wxw image. Thus, the sampling subspace 5 C R™ ^ “ 
is a space of images which take a constant value in each rxr block. 
Note that dim5 = w/r. The projection Sf = B^BJf of f on 5 
is obtained replacing the values in each of its r x r blocks by their 
average. Our goal is to estimate f from the input signal Sf. We know 
that the DCT captures most of the energy of natural images into 
a first few low frequency coefficients. Thus, a reasonable guiding 
subspace T is a space of images which are bandlimited to the lowest 
k X k frequencies. The projector T for this subspace is simply 
a low pass filter which sets the higher frequency components of 
the image to zero. The projector T can also be decomposed as 
BrB^. Here B^f involves taking the DCT of f and setting the 
high frequency coefficients to zero whereas Br converts these DCT 
coefficients to spatial domain to get a low frequency image. In our 
experiments, we study the effect of dimT = k x k on the quality 
of the reconstruction. We define fcscaie = (w/r)/k which compares 
the dimensionality of the sampling and guiding subspace. The value 
fcscaic < 1 corresponds to an undersampling problem, while fcscaie > 
1 corresponds to an oversampling scenario. A shorthand fd is used 
to denote the low resolution image BJf and tdu to denote the 
projection Sf. We also consider the scenario where the samples 


are contaminated by noise, i.e. = BJf -b e, where e is i.i.d. 
Gaussian noise. As a result, the input image becomes fd^ = 

B. Approaches under study 

We compare four reconstruction approaches, namely, the consis¬ 
tent reconstruction L, the generalized reconstruction fg, the regular¬ 
ized reconstruction fa and the minimax regret EH reconsh'uction 
fm = Tfdu- The consistent reconstruction L is calculated as L = 
x-bfdu, where x is the solution to problem S^T^x = —S^T^fd„ 
obtained using the CG method. 

The generalized reconstruction fg is computed using three dif¬ 
ferent implementations. In the first implementation, we solve the 
problem B^SBry = B^fju using CG to obtain y. The final 
reconstruction is then given by fgi = B^y. The second imple¬ 
mentation uses the projector T instead of the sampling operator 
B^, and the reconsh'uction fg 2 is the CG solution to the problem 
TSTf = Tfdu. In the third implementation, f^ is supposed to 
be available, and the generalized reconstruction is then computed 
by fg 3 = Tfc. Mathematically, it can be proved that all these 
implementations would produce identical reconstructions when the 
CG algorithm fully converges. However, these methods are algo¬ 
rithmically distinct and may converge at different rates as shown in 
the tests later. 

The regularized reconstruction fj., as posed in (|^, can be 
computed by solving (S -b pT''')f = fdu via CG. If fc and 
fg are available, we can simply take the convex combination 
fa = afc + (1 — ce)ig with p — (1 — a) /a. Because of Theorem[^ 
we have T = fa. Although these two solutions are mathematically 
equivalent (upon full convergence of CG), they are not similar 
algorithmically and exhibit different behavior and robustness against 
noise, when a small fixed number of CG steps is used in our tests. 

C. Experiments and observations 

We conduct four sets of experiments to study different as¬ 
pects of the reconstruction methods such as the effect of un¬ 
der/oversampling, effect of noise and convergence behavior. In our 
example, Aim'H = 256 x 256, dim 5 = 128 x 128. 

1) Experiment 1: In the first experiment, we take a noise-free 

signal fdu as input and observe the peak signal to noise ratio 
(PSNR = 201ogj^g reconstruction for different methods 

as the value of fcscaie (i.e. amount of under/oversampling) varies. 
For computing fa, we first fix a = 0.7. Fig.j^a) shows the plot of 
PSNR against fcscaic- We observe that in the undersampling regime, 
i.e. when fcscaie < 1, fc equals fg and performs better than fm- In 
case of oversampling, however. Fig. ia) demonstrates that T offers 
better PSNR than fg which, in turn, performs better than fm- This 
behavior is due to the fact that the sampling is noise-free, thus the 
method that keeps the samples unchanged is expected to perform 
better. The effect of a on the reconstruction quality is illustrated 
in Fig. I^b). Once again, we observe that as a increases (i.e. the 
samples are trusted more), the reconstruction quality improves. 

2) Experiment 2: In this experiment, we assume that the input 
fdu = Sf -b e is noisy, where e is i.i.d. Gaussian with zero mean 
and variance 0.001. We first focus on the performance of fa as 
a varies in case of oversampling by a factor fcscaie = 4. From the 
results shown in Fig. [^a), the best reconstruction is obtained with 
a = 0.7. This observation agrees with the theoretically suggested 
optimal value aojit = 1 — ||e||^/||fg — fc||^ = 0.7. Next, we analyze 
performance of fg, T, fm, and fa=o.7 for different values of fcscaie. 







(b) /^scale — 4 

Fig. 2: Effects of fcscaie and a on noise-free reconstruction 




(a) ^scale — 4 

Fig. 3: Effects of fecaie and a on noisy reconstruction 


in Fig. I^b). In contrast to the previous noise-free experiment, we 
notice that fc cannot always outperform fg when noise is present. 
The reconstraction fc only performs better than fg in the heavy 
oversampling regime, in this example, with fcscaie > 2.5. This 
observation indicates that, in case of slight oversampling, the noise 
filtering effect of the projection on guiding subspace offsets the loss 
due to sample inconsistency. For heavy oversampling, the sample 
consistency requirement is more important. We also observe that 
fc which is a weighted combination of fc and fg, can outperform 
both fc and fg for fcscaie > 1.5 for this example image, since f^ 
offers some noise suppression, while not deviating much from the 
consistency requirement. Fig. gives an example of a noisy input 
image and corresponding reconstructed images, demonstrating the 
advantages of fa with a = 0.7 vs. the traditional fg, fc, and fc. 

3) Experiment 3: In this experiment, we investigate the relation¬ 
ship between fa and E- in case of noisy inputs. Numerical results 
confirm that if the parameter p or a is known beforehand and fixed, 
the two approaches, despite having different implementations, give 
identical reconstructions. However, if the parameter p or a needs to 
be determined on the fly, the reconstruction E is clearly favorable 
compared to E in terms of computation complexity. For determining 
the whole set of solutions {E}, for a G (0,1), only one least 
squares problem needs to be solved, to compute E. All other can¬ 
didate solution points can be calculated by aE -b (1 — ajTE, since 
fg = Tfc. In contrast, search through the full set of {E}, for the 
optimal p G (0, oo), one least squares problem needs to be solved 
for each candidate solution, which may not be computationally 
feasible. 

4) Experiment 4: In this experiment, we compare the perfor¬ 
mance of the reconstruction methods in terms of the number of CG 
iterations. As described before, computing fg can be performed in 
three different ways, represented by fgi> fg 2 , and fg 3 - In Fig. we 
compare the three implementations with number of CG iterations, 
Maxiter, set to 1 and 2 applied to the noisy input. We observe that 
El = fg 2 in both cases. Although fgs is different when the number 
of iterations is 1, as seen in Fig.[^a), the difference becomes very 
minor when the number of iterations equals 2. This observation also 





(c) E, PSNR=22.00dB (d) E=o. 7 , PSNR=22.88dB 
Fig. 4: Reconstruction results with noisy inputs, fcscaie = 4 




Fig. 5: Performance of different implementations of E- 


holds for noise-free inputs. Since E has three implementations, E 
can also have different corresponding implementations, given by 
Ei = aE + (1 — a)fgi with i = 1, 2, 3. The performance of all the 
reconstruction methods with different implementations is shown in 
Fig. § All the algorithms are configured to use Maxiter number 
of CG iterations, except fm since it does not need least squares. 
The reconstruction E 2 is omitted as it is always equal to Ei. We 
observe that Ei = E2 performs better than E3. In case of heavier 
oversampling, E is more favorable than E. 




(a) Maxiter = 1 (b) Maxiter = 2 

Fig. 6: Reconstructed image qualities 


Conclusion 

The proposed frame-less iterative algorithm allows efficiently 
reconstructing noisy signals with desired properties described by 
a guiding subspace, which can be given by an approximate pro¬ 
jector. Numerical examples for image magnification demonstrate 
the advantages of our method, compared to the traditional sample 
consistent and pure guided methods. The suggested methodology is 
expected to be effective for a wide range of signal reconstruction 
applications, in video and speech processing, and machine learning. 
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